Generality: A New Criterion for Measuring Generality of Documents

Most information retrieval systems, including Web search engines, use similarity ranking algorithms based on a vector space model to find relevant information in response to a user’s request. However, the retrieved information is frequently irrelevant, because most of the current information systems employ index terms or other techniques that are variants of term frequency. In this paper, we propose a new criterion, “generality,” that provides an additional basis on which to rank retrieved documents. We compared our generality quantification algorithm with human judges’ weighting of values to show that the developed algorithm is significantly correlated.